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® Current television transmission standards in 
Europe and other parts of the world operate v^ith a 
50H2 display refresh rate. The process of refreshing 
a CRT display causes a change of light intensity 
with time, called flicker. One method of removing the 
visibility of this flicker is to refresh the display at a 
higher frequency. The data in the additional fields 
may be generated from the original input signal 
when the images are static. When the images move, 
the subjective effect In each case is one of a double 
image. This is due to the data in the new fields 
being displayed in the wrong temporal position, and 
when the eye tracks the motion across the screen, a 
double image of one form or another results. 
This appearance of a double image around moving 
objects may be reduced or removed by generating 
motion compensated 100Hz fields with the object in 
the correct position. When, however, the results of 
this technique are applied to motion compensated 
upconversion, objects or parts of the picture which 
fail outside the range of measurable velocities ap- 
pear on the upconverted display with a very strong 
double image, resulting from the display of informa- 



tion in the wrong temporal position generated by 
uncompensated or 'fallback processing*. 
According to the invention, the subjective strength of 
the double image is reduced by low pass filtering in 
a horizontal direction the components in the new 
fields of the upconverted signal which cause this 
double image. 
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The present invention relates to a nriethod and 
to an apparatus for motion compensated interpola- 
tion of intermediate fields or frames. 

Background ^ 

Current television transmission standards in 
Europe and other parts of the world operate with a 
50Hz display refresh rate. One complete image, or 
frame, is transmitted every l/25th of a second. to 
Each frame is composed of two half-images, or 
fields, each one of which is transmitted every 
1/50th of a second. Each field consists of either the 
odd or even lines of a frame. 

The process of refreshing a CRT display causes a is 
change of light intensity with time, called flicker. At 
50Hz, the human eye is still sensitive to this flicker, 
and on bright scenes and/or large displays this 
flicker is noticeable. One method of removing the 
visibility of this flicker is to refresh the display at a 20 
higher frequency. This increase in field rate may 
be by any arbitrary amount as long as a frequency 
is chosen which is sufficiently great that the sen- 
sitivity of the eye is very low. The frequency used 
as an example in this description is 100Hz, a 25 
doubling in frequency which results in a relatively 
simple hardware implementation, although in princi- 
ple, any frequency may be used and 75H2 has 
also been contemplated. 

With the example frequency of IOOH2. there are 30 
two scanning patterns which may be used, these 
are known as AABB and ABAS, where the letters A 
and B refer to the positions of the odd and even 
fields F. The arrangements of the two types of 
scanning is shown in Figures 1A and 1C. Lines L 55 
which belong to the original input signal are 
marked by a V and new lines which must be 
created are marked by a 'x'. 

The data in the new fields may be generated from 
the original input signal as illustrated by the arrows 4o 
in Fig. 1 . This presents an acceptable picture qual- 
ity when the images are static. 

When the images move, the subjective effect 
in each case is one of a double image. This is due 
to the data in the new fields being displayed in the 45 
wrong temporal position, and when the eye tracks 
the motion across the screen, a double image of 
one form or another results. This effect is shown in 
Rgures IB and ID, the letter of which shows 
'mouth teeth* artefacts. 5o 
This appearance of a double image around moving 
objects may be reduced or removed by generating 
the new IOOH2 fields with the object in the correct 
position. This is done by measuring the velocity 
with which the object is moving and then interpolat- 55 
ing the new field using information from the correct 
position (i.e. from the object) in the source fields. 
This technique is called motion compensated inter- 



polation, and in this instance is applied to 
frame/field rate upconversron. but could equally 
well have other uses, for example in the generation 
of an improved slow motion video signal, or in 
standards conversion. 

One technique of motion estimation is known as 
block matching. Typically, the current field or frame 
of the picture is divided into rectangular blocks. For 
each block, a search is made over overlapping 
blocks in the previous field or frame to find the one 
that matches the current field block the best ac- 
cording to some criterion, typically mean-square or 
mean absolute error. The relative position of the 
two blocks gives the displacement, or motion, vec- 
tor i6r the current block. 

Invention 

It is one object of the invention to disclose a 
method of improved fallback processing. This ob- 
ject is reached by the method disclosed in claim 1 . 

It is a further object of the invention to disclose 
an apparatus which utilises the inventive method. 
This object is reached by the apparatus disclosed 
in claim 5, 

The above described motion estimation princi- 
ple holds whether single or double sided block 
matching is used. Double sided block matching 
and its advantages are described in EP-A- 
93402187 of the applicant. 

As block matching is computationally intensive, 
real-time hardware implementations of this tech- 
nique implement some form of parallel processing. 
The most common form is to have one block 
matching processor per candidate motion vector. 
An example hardware implementation for a single 
sided block matching scheme is shown in Rg. 2. 
This implementation allows the measurement of 
velocities of up to ±2 pixels or lines in the horizon- 
tal and vertical directions. Each processing element 
matches, in parallel, the cun-ent block with those in 
the previous field or frame displaced by ±2 pixels 
or lines. 

In order to give good results in all cases, the block 
search range should cover all possible motion vec- 
tors which may be present in the incoming signal. 
As the number of processing elements is directly 
related to the complexity, and hence cost, of the 
hardware implementation, there is some incentive 
to minimise the number of candidate vectors which 
must be checked. 

As described in EP-A-93402059 of the applicant, 
one method of reducing the hardware cost is to 
cover only a limited range of possible motion vec- 
tors within the image, and to cover some of this 
range with reduced precision. 
An example of this related to motion compensated 
frame/field rate upconversion is shown in Fig. 3 
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where is a site of a motion vector processor In 
the two-dimensional velocity space VyA/n. The mo- 
tion estimation is performed (within area AHP of 
high precision) with a good precision of small or 
slow motion and as the velocity of the object in- 5 
creases, the precision of the estimation decreases 
(within area ALP of low precision). Finally, there Is 
a limiting velocity (uncompensated region UCR) 
beyond which no motion estimation Is performed 
and where compensation is therefore Impossible io 
and a fallback processing Is required Instead. It Is 
with respect to this uncompensated region that the 
present invention relates. In addition, the invention 
is applicable to both scanning types shown in Fig. 
1A and Rg. IB. /s 

As already mentioned, economic factors rule 
against the implementation of a motion compen- 
sated upconversion scheme which is capable of 
applying motion compensated upconversion for all 
possible velocities of objects which may be con- 20 
tained within the incoming television signal. The 
fallback processing treats the case where the ve- 
locity of an object, or pan across a scene exceeds 
the maximum velocity supported by the motion 
estimation system. Fig. 4 shows one possible ef- 26 
feet at the edge of a moving object as the object 
accelerates through the range of velocities shown 
in Fig. 3, when viewed after a typical upconversion 
process. The object starts from rest (zero velocity 
2V, Fig. 4a). As the object accelerates, the error in 30 
the motion is well compensated (integer pixel ve- 
locity IPVH and non-integer pixel velocity NIPVH in 
region AHP of high precision) until the region of 
lower precision measurement Is reached (non-in- 
teger pixel velocity NIPVL in region ALP of low 35 
precision), which can be considered acceptable as 
the human visual system does not register such 
small defects on an object moving with a relatively 
high velocity. What is unacceptable, is the abrupt 
change as the edge of the motion compensated 4o 
velocity region is crossed (velocity VOSR outside 
of supported region AHP and ALP). The Image will 
abruptly change from the appearance in Fig. 4b, 4c 
or 4d to that of Fig. 4e. 

Referring back to Fig. 1A and 1C, it can be seen 45 
that this will result in a sudden change in parts of 
the image, from a high quality single moving image 
to a low quality double image, with or without 
•mouse-teeth* depending on the type of upconver- 
sion in use. It is the purpose of the Inventive so 
fallback processing to Improve the subjective ap- 
pearance of images moving with a velocity too 
large to be correctly compensated. 
It can be seen from Fig. 1 that the images which 
are in the wrong temporal position are those of the 55 
new fields (marked by 'x's). Considering the case 
in Fig. 1A of the AABB type scanning (ABAB is 
similar), the complete image is already present in 



the correct temporal position in the original fields. 
When motion compensated processing is impos- 
sible, the new fields may be generated by non- 
compensated methods, such as repeating the origi- 
nal fields, or by generating a progressive scan 
image and delaying alternate lines by 10ms. The 
subjective appeareince of the double image is than 
significantly improved by heavily low pass filtering 
the new fields In the horizontal direction. The filter- 
ing must be active over a region comparable in 
size to the double Image. 

The effect of this filtering is to remove any signifi- 
cant detail from the newly interpolated fields. The 
filtering process blurs the fields which are dis- 
played with information in the wrong temporal posi- 
tion. This blurring in every second field of a IOOH2 
video signal is acceptable as the human visual 
system does not register It on an object moving 
with high velocity. As detail Is only present in fields 
with information present in the correct temporal 
position, the subjective strength of the double im- 
age is greatly reduced, resulting In a greatly im- 
proved subjective picture quality. In this Implemen- 
tation, the fallback processing is applied to the new 
fields when a motion vector cannot be determined, 
that Is normally when the velocity lies in the un- 
compensated region UCR. 

In principle, the inventive method is suited for 
motion compensated interpolation of intermediate 
fields or frames of a input video signal using pixel 
block motion vectors, whereby in case of related 
motion velocities extending a preselected range or 
in case said motion vectors cannot be determined 
a fallback Interpolation is performed in which pixel 
values of the intermediate fields or frames only are 
lowpass filtered horizontally and whereby the filter- 
ing is active over a region comparable in size of 
double image artefacts caused by the missing mo- 
tion compensated interpolation. 

Advantageous additional embodiments of the 
Inventive method are resulting from the respective 
dependent claims. 

In principle the Inventive apparatus for motion 
compensated interpolation of intermediate fields or 
frames of a input video signal using pixel block 
motion vectors Includes: 

- motion compensated interpolation means and 
fallback interpolation means which receive 
said Input video signal and the output pixel 
values of which are selected In switching 
means in relation to said motion vector or its 
horizontal component, whereby in said fall- 
back interpolation means pixel values of the 
Intermediate fields or frames only are 
lowpass filtered horizontally and whereby the 
filtering is active over a region comparable In 
size of double Image artefacts caused by the 
missing motion compensated interpolation; 



EP 0 648 046 A2 



- motion estimation means which calculate 
from said input signal, or motion information 
extractor means which extract from the trans- 
mitted input signal data stream, said block 
based motion vector and/or its horizontal s 
component, whereby in case of said motion 
vector or its horizontal component extending 
a preselected range or in case of said motion 
vector cannot be determined said fallback 
interpolation means are selected by said io 
switching means. 
Advantageous additional embodiments of the 
inventive apparatus are resulting from the respec- 
tive dependent claims, 

75 

Drawings 

Preferred embodiments of the invention are 
described with reference to the accompanying 
drawings, which show in; 20 

Fig. 1 AABB and ABAB scanning for upcon- 
version and resulting artefacts; 

Fig. 2 typical architecture for one-sided 
block matching; 

Fig. 3 example configuration of reduced cov- 25 
erage of motion vector processors; 

Fig. 4 appearance of a moving object accel- 
erating horizontally through the veloc- 
ity space of Fig. 3; 

Fig. 5 fallback processing low pass filter; 30 

Fig. 6 simplified block diagram of inventive 
apparatus. 

Preferred embodiments 

35 

Fig. 2 shows a possible architecture for one- 
sided block matching motion estimators with a 
range of ±2 pixels per field horizontally and verti- 
cally, to integer accuracy. The architecture is 
based on processing elements PE. one per can- 4o 
didate motion vector. Each processing element PE 
accumulates the errors between the pixels (which 
form a group related to a candidate block position 
and vector, respectively) an^iving at its two inputs, 
storing partial results in order to arrive at a total 45 
error for each block. The processing elements are 
connected together by means not shown in the 
Rgures In order to find which one yields the mini- 
mum error for each block. The processing ele- 
ments may calculate the minimum absolute value so 
or the minimum squared error. 
AH the processing elements PE have a common 
input IN, corresponding to the current block. The 
relative displacements between the pixels in the 
search window are provided by a network of line 55 
LD and sample SD delays that follow the field 
delay FD connected to input INP. 



The fallback processing low pass filter in Fig. 5 
passes the input signal (i.e. the 'x' pixels in the 
lines of Fig. 1) through a chain of one horizontal 
sample delays T. The respective delayed output 
pixel values and the pixel values of the input signal 
IN are multiplied by the filter coefficients 1/16, 
2/16, 3/16. 4/16. 3/16. 2/16, 1/16 in multipliers X.' 
The multiplier outputs are combined in adder ' + ' 
to form the final output pixel values OUT for the 
intermediate fields. 

In Fig. 6 the 50/60Hz input signal IN passes 
through a motion compensated interpolation circuit 
MCI or a fallback interpolation circuit FBI and a 
switch SW to the 100/120Hz output. Circuit ME can 
be a motion estimator which calculates related mo- 
tion vectors MV from the input signal IN and con- 
trols switch SW by detecting strongly moving pic- 
ture areas in which no motion estimation can be 
performed. This information can also be transmit- 
ted if the motion estimator is located within a 
transmitter and the circuit of FHg. 6 is part of a 
receiver, e.g. a TV receiver. Then, circuit ME may 
extract the motion vector MV and the information 
on non-motion estimated picture areas from the 
transmitted input signal data stream. The motion 
information is used in interpolator MCI to perform 
the motion compensated interpolation of the inter- 
mediate fields. Switch SW can also operate as a 
soft switch smoothing the MCI/FBI transition at the 
outer boundary of area ALP. Circuit ME may in- 
clude a circuit according to Rg. 2. Interpolation 
circuit FBI may include a filter according to Fig. 5. 

In the case of a uniform moving edge, the 
motion vector is only valid in the direction per- 
pendicular to the edge. In the example given, the 
fallback processing is limited (for reasons of cost) 
to horizontal filtering. Therefore, one implementa- 
tion could be to limit the application of the fallback 
mode to cases when the horizontal component of 
the velocity only is out of range (i.e. in the case of 
edges, operate solely on vertical edges). 
Another possibility, as described in the accom- 
panying patent application EP93 402506 of the 
applicant could be to control the application of the 
fallback mode processing through some intermedi- 
ate form of vector conditioning, such as a measure 
of confidence. 

The invention can also be used for standards 
conversion, bit rate reduction, digital HDTV, digital 
VCR. digital video disc player and MPEG1 or 
MPEG2 devices (lEC/ISO standard). 

Claims 

1. Method for motion compensated interpolation 
(MCI) of intermediate fields or frames (x) of a 
input video signal using pixel block motion 
vectors, characterised in that in case of re- 
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lated motion velocities (Vh. Vy) extending a 
preselected range (AHP, ALP) or in case said 
motion vectors (MV) cannot be determined a 
fallback interpolation (FBI) is performed in 
which pixel values of the intermediate fields or 5 
frames (x) only are lowpass filtered (T, X, +) 
horizontally, whereby the filtering is active over 
a region comparable in size of double image 
artefacts (Fig. 1A, Fig. IB) caused by the 
missing motion compensated interpolation. to 

Method according to claim 1. characterised 
in that said fallback interpolation (FBI) is per- 
formed only when the horizontal component 
(Vh) of the motion velocity is out said preselec- is 
ted range. 

Method according to claim 1 or 2. charac- 
terised In that said fallback Interpolation (FBI) 
is performed in relation to a measure of con- 20 
fidence of said motion vector (MV). 

Method according to any of claims 1 to 3, 
characterised \n that a soft switching (S\AO is 
performed between said motion compensated 26 
interpolation (MCI) and said fallback interpola- 
tion (FBI). 

Apparatus for motion compensated interpola- 
tion (MCI) of intermediate fields or frames (x) 30 
of a input video signal using pixel block motion 
vectors, related to a method according to any 
of claims 1 to 4 and including: 

- motion compensated interpolation means 
(MCI) and fallback interpolation means 35 
(FFI) which receive said input video sig- 
nal (I) and the output pixel values of 
which are selected in switching means 
(SW) in relation to said motion vector 
(MV) or its horizontal component (Vh), 40 
whereby in said fallback interpolation 
means pixel values of the intermediate 
fields or frames (x) only are lowpass fil- 
tered (T, X, +) horizontally and whereby 

the filtering is active over a region com- 46 
parable in size of double image artefacts 
(Fig. 1A. Fig. 1B) caused by the missing 
motion compensated interpolation; 

- motion estimation means (ME) which cal- 
culate from said input signal (IN), or mo- so 
tion information extractor means (ME) 
which extract from the transmitted input 
signal data stream, said block based mo- 
tion vector (MV) and/or its horizontal 
component (Vh), whereby in case of said 55 
motion vector or its horizontal component 
extending a preselected range (AHP, 
ALP) or in case of said motion vector 



cannot be determined said fallback inter- 
polation means (FBI) are selected by 
said switching means (SW). 

6. Apparatus according to claim 5. characteris- 
ed in that said switching means {SW) are soft 
switching means. 

7, Apparatus according to claim 5 or 6, charac- 
terised in that said apparatus is part of a 
digital TV or HDTV receiver or a digital VCR or 
a digital video disc player. 
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@ Current television transmission standards in 
Europe and other parts of the world operate with a 
50 Hz display refresh rate. The process of refreshing 
a CRT display causes a change of light intensity 
with time, called flicker. One method of removing the 
visibility of this flicker is to refresh the display at a 
higher frequency. The data in the additional fields 
may be generated from the original input signal 
when the images are static. When the images move, 
the subjective effect in each case is one of a double 
image. This is due to the data in the new fields 
being displayed in the wrong temporal position, and 
when the eye tracks the motion across the screen, a 
double image of one form or another results. 
This appearance of a double image around moving 
objects may be reduced or removed by generating 
motion compensated 100Hz fields with the object in 
the correct position. When, however, the results of 
this technique are applied to motion compensated 
upconversion. objects or parts of the picture which 
fall outside the range of measurable velocities ap- 
pear on the upconverted display with a very strong 
double image, resulting from the display of informa- 



tion in the wrong temporal position generated by 
uncompensated or 'fallback processing'. 
According to the invention, the subjective strength of 
the double image is reduced by low pass filtering in 
a horizontal direction the components in the new 
fields of the upconverted signal which cause this 
double image. 
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